Mega Rewards Campaign

Executive Summary

After extensive research in 2012, it was determined that the most profitable customer groups of OLG loyalty card holders are Segments 1,2,3,4 and 5 (shown below). In order to reward these customers OLG has launched Mega Rewards campaign offering various prizes including restaurant gift certificates to be used at OLG properties, HBC gift certificates, “OLG slot machine cash” that will enable customers to play up to $30 worth of play at OLG slot machines for free and 2000 points to be added to patron’s OLG gold, silver or bronze loyalty card point balance. Promotion was available at Woodbine and Ajax properties from Jan 1, 2013 to Jan 10 - 2013.

The campaign, which cost a total of $ 92,920 was able achieve a ~13% return on investment (ROI), $1.3 million in increased slot revenue, and a response rate of 65 %. Analysy concludes that segment 5 was by far the most successful customer group. This segment is composed of females aged 35-40 years old who were given 2000 loyalty card points (valued at $20). It was suspects that this segments success was primarily derived from (1) the fact that segment 5 already generates the most pre-campaign revenue, and (2) they were issued points on their loyalty cards.

Location information provided from the postal code information was used to derived several maps that reveal the spatial effects and features of the campaign. It was found that the regions closest to the OLG facilities of Ajax and Woodbine had the lowest response rates (0 - 0.2) compared to the rest of the regions (0.6 - 1.0). It is imperative to understand why these response rates are so much lower before the next campaign.

By plotting the cumulative coupon claims over time we are able to see when and at what rate different segments and groups of people claim the rewards. The overall redemption rate was linear until a threshold of around 3000 / 5000 claims were met. All five customer segments claimed their coupons at the same rate. Interestingly, the coupons valued at $20 (including those distributed as points) actually reclaimed nearly 100% of the issued coupons, compared to the ~50% claimed by other dollar values.

Several recommendations for future campaigns are explained in detail in the concluding section. The most important modification is to redesign the distribution of coupons to customer segments. To confidently understand the performance of future marketing campaigns and the success of individual coupon rewards, OLG must distribute all coupon rewards to all segment groups evenly, and utilize control groups in the study. It is further recommended to test longer promotion periods, and utilize more data in the analysis.

Campaign Costs Breakdown

Overview

• Creative (graphic design, marketing agency fees): $ 35000
• Postage: $ 2500
• Total coupon value offered: $ 85000
• Redeemed coupon costs: $ 55420

  ---

  Total Campaign Cost: $ 92920

Performance Metrics Analysis

Overall Campaign Metrics

Total Incremental Slot Wins	Total Incremental Slot Wins (%)	Total ROI	Total Response Rate
$ 1303300	1.1333 %	13.026 %	65.24 %

Campaign slot winnings are statistically different than pre campaign slot winnings.

Difference of Means t-test : p < 0.005

Segmented Campaign Metrics

A) Segment Groups

First we visualize the distribution of coupons per segment. In this campaign one type of coupon was given to each different segment, with no overlaps. This methodology makes it difficult to derive many insights into the performance of the individual campaign segments, since we cannot determine if the campaign was successful because of the different coupons or simply the behaviour of the segments themselves.

The table below is a simple summary table of the campaign efforts.

Segment	Demographic	Coupon Code	Coupon Value	Coupon Type	Frequency
1	Male, 25-30 years old	A	$5	Gift Card	1000
2	Male, 30-35 years old	B	$10	Gift Card	1000
3	Female, 25-30 years old	C	$20	Cash	1000
4	Female, 30-35 years old	D	$30	Cash	1000
5	Female, 35-40 years old	E	$20	Points	1000

We can see below that the individual segments are in fact very different. Therefore, we cannot be conclusive regarding the effect of the different types of coupons, or decide things like ‘which coupons work best?’. However, clearly we can see that all the coupons were successful in generating additional revenue. For each of the segments, there is a statistically significant increase in slot revenue from the effect of the campaign. Though the performance is clear in this test dataset, it would be strongly recommended to include a control group in future marketing campaigns. This way we can be certain that the results are due to the campaign, and not some other phenomenon.

Campaign slot winnings are statistically different than pre campaign slot winnings.

Technically cannot compute with given data.

Summary Table

Segment	Slot.Increase.Dollars	Slot.Increase.Percent	ROI	Response.Rate
1	65200	1.100	19.23	0.652
2	131000	1.100	19.12	0.655
3	261600	1.133	19.06	0.654
4	325500	1.125	15.70	0.651
5	520000	1.160	39.12	0.650

• Return on investment derived from equal distribution of campaign costs to each segment - use with caution.

The results for the increase in slot winnings per segment are presented above, along with several other metrics. Every segment has increases in revenue from the campaign, similar response rates, and positive return on investment. The response rate is up ~ 20% since January 2012, but still falls short of 2012 spring and summer campaigns; April - 78% and June - 85% . Assuming an equal distribution of cost, we can see that the most ROI is gained from segment 5 at 39%.

Recall that segment 5 is composed of females, 35-40 years old who were given 20$ in loyalty card points. This segments ROI is vastly different than the other segments, and is likely due to the use of points, rather than cash or gift cards. Points (stored on cards) are generally considered less tangible than cash, which tends to make them ‘easier’ to spend. A study from MIT in 2000 found that credit cards can increase willingness to pay by up to 100% compared to using cash. In addition to the type of coupon used, the factors that contributed to segment 5’s exceptional ROI were having the largest percentage increase in slot revenue, and interestingly the lowest response rate (marginally). It would be expected to see more variation in the response rates, perhaps the relatively low response rate for segment 5 is due to the delivery of the campaign, ie: email vs. mail vs. phone.

The other segments also perform well. However, we see that the lowest ROI is from segment 4, who we would expect to generate more ROI because they are the second highest spending group (during the time of the study). For now we cannot be certain if the sub-par ROI is due to the coupon used, or due to unknown factors about the particular segment tested during the study interval. To determine the reason behind this we would have to test this coupon on several other segments and measure the response, and also include a control group. With the given information we can make two plausible conclusions:

1. Players from segment 4 simply didn’t play as much during the campaign.
2. Players from segment 4 do not care about the promotion or did not receive the coupon.

Lastly, with more detailed analysis in the future, it might be the case that there is a threshold for ROI from some coupon promotions. For example, it could be that after $20, there are no significant gains in ROI. This would allow for cost savings by not offering overly costly rewards.

The results of the coupon type on slot performance from the campaign are reported below. Note that the results are slightly biased because of how the coupons were distributed. For example: the ROI for the Points type is the same as segment 5, because they are both the same mutually exclusive group.

B) Coupon Type Groups

Coupon.Type	Slot.Increase.Dollars	Slot.Increase.Percent	ROI	Response.Rate
Cash	587100	1.129	17.01	0.6525
Gift Card	196200	1.100	19.04	0.6535
Points	520000	1.160	39.12	0.6500

• Return on investment derived from equal distribution of campaign costs to each segment - use with caution.

C) Gender Groups

We can do the same sort of analysis with Gender (or any of the other variables), though again the results are potentially misleading due to the segmentation/coupon distribution method.

Gender	Slot.Increase.Dollars	Slot.Increase.Percent	ROI	Response.Rate
Female	1107100	1.142	23.28	0.6517
Male	196200	1.100	19.04	0.6535

• Return on investment derived from equal distribution of campaign costs to each segment - use with caution.

Difference of Means t-test, all p < 0.005

• Male Campaign slot winnings are statistically different than pre campaign slot winnings.
• Female Campaign slot winnings are statistically different than pre campaign slot winnings.
• Male Campaign slot winnings are statistically different than Female campaign slot winnings.
• Male Pre-Campaign slot winnings are statistically different than Female pre-campaign slot winnings.

Location Data

The information provided using the postal codes provides insight into the spatial effects of the campaign. It also turns out that in this test dataset, the postal code information is the most variable of all the data provided.

Concerning the distribution of the coupon redemption, it was found that the northern postal codes (L0G, L3Z, L3X, and L3Y) all redeemed their coupons in Ajax, which for some regions is considerably farther away than Woodbine Racetracks. This may be due to specific features of the Ajax location that differ from Woodbine, more research is needed to understand why Ajax attracts these communities.

Map 1

The map below depicts the total coupons sent to the different postal codes. We see a moderate trend whereby more coupons were given to customers that were farther away (possibly intended?). In any event this is probably a good strategy but should be backed up by data showing that these customers either:

• Play less, such that we want to entice them to play.
• Play a lot already, such that we want to reward the biggest customers.

Map 2

The second map depicts the average response rate of each FSA. Interestingly, the response rates for the two FSA’s closest to either OLG facility show the lowest response rates. These locations are driving down the average response rate we saw in the previous analysis ( ~65% ). There is no clear reason for these locations to have such poor response rates without further information. Immediate action should be taken to understand this effect and improve the response rates. There are potential opportunities for increase revenue and better serving of the customer.

Map 3

In the final map below we see the ROI by FSA region. It is now clear why the ROI is low for the two regions closest to either facility. This is likely because:

• The fewest coupons were sent here, and…
• These areas had the worst response rates.

However, the ROI is already scaled by the number of coupons sent to each location, so the culprit here for low ROI is likely the lower response rates. We can see that only two postal codes had both low response rates, and low coupons sent out; the ones overlapping the facilities ( L1Z and M9W )

Summary Table

Below is a summary table of the results in tabular form. We can use it in conjunction with any of the labelled maps to see where the postal codes are located.

FSA	Slot.Increase.Dollars	Slot.Increase.Percent	ROI	Response.Rate
L0G	172800	1.133	22.858	1.0000
L1Z	2700	1.129	-15.026	0.0667
L3X	101200	1.133	23.218	0.9236
L3Y	198000	1.133	22.743	1.0000
L3Z	556800	1.133	22.540	0.6153
L4J	94200	1.133	23.337	0.8172
M3H	33200	1.133	32.927	1.0000
M3M	17200	1.132	28.449	0.1888
M3N	28000	1.133	41.795	1.0000
M6S	16000	1.133	26.314	0.1005
M8W	81200	1.134	23.520	0.7153
M9W	2000	1.133	-4.632	0.0847

• Return on investment derived from equal distribution of campaign costs to each segment - use with caution.

Coupon Redemption Rates

The following plots depict the rates of coupon redemption over time by different variables. This is mostly provided as examples and talking points for the purposes of this assignment.

Recommendations

1. Improve campaign implimentation methods for coupon distribution

   This is the most imperative point to improve for the next marketing campaign. While we can somewhat conclude on the overall success of this campaign, the implementation methods are counter productive. We actually cannot safely conclude that this marketing campaign was a success, because we do not know that the improved slot performance was from the campaign alone and not some other factor, for example: we might expect an increase in spending from the first to second weeks in January due to ‘New Years’. The simplest way to test this is split our data into study & control groups. To create control groups, we simply leave out a known portion of our target segments from the campaign benefits, and then track both the study and control groups behaviour during the campaign. If the study segment significantly outperforms the control segment, we can be much more confident that the results are due to the marketing campaign.

   The second recommendation related to this section is to distribute the different coupon rewards to every segment (along with including a control group). This would make it possible to determine which marketing campaigns are most effective for each demographic segment. This would allow fine tuning of future campaigns to optimize revenue and customer satisfaction. Instead of testing each coupon reward on one random segment, a suggested configuration for the above described methodology would be:

• Split all \(N\) data into \(d\) demographic segments and \(n\) evenly distributed subgroups, where \(n\) is the number of different rewards. For a sample with 10 000 customers , 4 types of rewards , and 5 customer segments , each demographic segment would have 2000 test subjects for each reward, and 2000 test subjects in a control group.

2. Lengthen study duration

It would be wise to experiment with lengthening the study duration to see if we can acquire a higher response rate, and generate increased revenue by simply allowing a more flexible time-frame for customers to redeem their coupons.

Secondly, we should be using much more than 10 days worth of data to better understand the pre-campaign slot behaviour. There is opportunity for a much more sophisticated use of time series analysis for the marketing campaign. Using this added information we could begin to predict the success of future market campaigns before they have been implemented.

3. Record more variables

More good data is always better for analytics, there is certainly opportunity to include more micro information about the behaviour of the customers in response to the campaign. Some possible examples include:

• More historical gaming data
• New vs. long-term customers
• Unaggregated data sources
• Time data (hr:min:s)
• Behaviour of players who don’t redeem the coupons
• Dates of other promotions or events (horse races)

Next Steps

This section is dedicated to the work I would have done had there been more time or data.

1. Investigate relationship between census data and campaign response.
2. What percentage of people do the customer segments capture from the total population in the census tracts? ie. Where are there more potential customers that fit in our profitable segments.
3. Regression / Classification Analysis.

There is limited room or need for regression or classification analysis with this data because it has so little variation. My work on statistical models is well documented on my website.